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Speech to Text Captioning Digital Camera 



5 Technical Field of the Trivention 

The present invention relates generally to digital cameras and in particular the 
present invention relates to captioning speech in a digital camera. 



Background of the In ve ntion 

10 The field of photography has experienced significant advancements with the 

development of digital cameras. These cameras capture images and store the images as 
a digital file. Some cameras include features to allow for text to be superimposed on 
images. For example, a date or caption such as "1999 vacation" can be entered and 
superimposed on pictures. This text is typically used for several pictures and changes 

1 5 are relatively difficult. 

In a similar maimer, some cameras include a feature to capture speech. The 
speech is stored as a separate audio file fi*om the image file. The two files are usually 
named in a similar manner (different extension), such as 0021,jpg and 0021.mpa for 
respective image and speech files for picture 0021. Playback of the audio file is 

20 awkward on a device other than the camera. 

For the reasons stated above, and for other reasons stated below which will 
become apparent to those skilled in the art upon reading and understanding the present 
specification, there is a need in the art for a camera which allows for custom picture 
annotation using an audio input. 

25 



Summary of th e In ve ntion 
In one embodiment, a camera comprises a photo-sensitive array for capturing an 
image, a microphone, a memory, and a processor coupled to the photo-sensitive array, 
30 microphone and memory. The processor converts audio input provided by the 
microphone into text and stores the text in the memory, 

1 



In another embodiment, a camera comprises a photo-sensitive array for 
capturing an image, a microphone, a memory, and a processor coupled to the photo- 
sensitive array, microphone and memory. The processor converts captured audio input 
provided by the microphone into a digital text file and converts the captured image into 
5 a digital image file. The processor also stores the digital image file and the digital text 
file as a single composite digital data file in the memory, 

A method of operating a camera is provided which comprises activating a 
shutter of the camera to capture a Ught image, and converting the light image to digital 
image data. The method also includes activating an audio input, capturing audio input, 
10 and converting the audio input into text data. 



Brief Description of the Drawings 
Figure 1 is a block diagram of a camera according to one embodiment of the 
present invention; 

15 Figure 2 is an example photograph including a text message; 

Figure 3 is a flow chart of an example operation of the camera of Figure 1; and 
Figure 4 illustrates an altemate embodiment of a camera system of the present 
invention. 



20 Detailed Description of the Invention 

In the following detailed description of the preferred embodiments, reference is 
made to the accompanying drawings which form a part hereof, and in which is shown 
by way of illustration specific preferred embodiments in which the inventions may be 
practiced. These embodiments are described in sufficient detail to enable those skilled 

25 in the art to practice the invention, and it is to be understood that other embodiments 
may be utilized and that logical, mechanical and electrical changes may be made 
without departing fi*om the spirit and scope of the present invention. The following 
detailed description is, therefore, not to be taken in a limiting sense, and the scope of the 
present invention is defined only by the claims. 
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Figure 1 illustrates a digital camera 100 according to one embodiment of the 
present invention. The camera includes an input lens 102 (including a shutter) for 
directing light to a light sensitive array 104. In operation the shutter is activated to 
direct a light image onto the array. This array can be any photosensitive array, such as a 
5 charge coupled device (CCD), CCDs have dominated vision applications because of 
their superior dynamic range, low fixed-pattern noise and high sensitivity to light. The 
array is coupled to a processor 106. The processor can transfer the image captured in 
the array to memory 110. Numerous images stored in the memory can be output 
through output circuitry 1 12. A microphone 108 is coupled to the processor for 
10 capturing speech, as explained below. The processor can process the speech and then 
route the speech to the memory for storage. The term processor as used herein 
encompasses circuitry required to convert the light image to digital data. Further, the 
processor can include any control circuitry needed to operate the camera, including 
output operations. 

15 In operation, a user of the digital camera can activate a speech capturing 

operation using control 114. In one embodiment, control 1 14 comprises a switch which 
can be activated when speech is desired. Other types of control circuitry are 
contemplated, including speech recognition using the microphone and processor. In 
this embodiment, a pre-determined audio command is used to "wake-up" the camera 

20 speech capture mode. When the processor is in the speech capture mode, output speech 
signals provided by the microphone are converted to text using the processor. As such, 
the processor operates a speech translation operation. This operation is performed by 
executing a set of instructions which instruct the processor to convert the received 
speech. The text is then used to add captions to the photograph which was taken most 

25 recently. 

Referring to Figure 2, a sample photograph is provided. The photograph 
includes a text message which describes the photograph as "sunset in the mountains 
during vacation of 1999". This text message is intended to illustrate the flexibility of 
the present invention to customize text on each photograph. The processor 106 can 
30 store the text message as either a separate text file (text data), or combine the 
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photograph data (digital image) and the text data as one digital file. As separate files, 
the speech can be stored as an audio file, or converted to text. By combining the files, 
unwanted separation of the photo and text can be avoided. Further, the text can be 
separately stored in parallel to the photo/text file to provide a level of redundancy. 
5 Figure 3 provides an illustration of the operation of the digital camera. A user 

takes a photograph at 200 by activating the shutter. The photograph image is stored at 
202. The user initiates audio speech input at 204, and the speech is converted to text at 
206. Alternatively, a compresses audio file is stored of the speech at 208. The text is 
combined with the image at 210 to provide a common file which is stored in memory. 

10 Alternately, the text is stored as a separate file at 212. 

Figure 4 illustrates a camera system of the present invention. The system 
includes a digital camera 300 and an external processor 310. The camera is 
substantially similar to the camera of Figure 1 except the speech to text processor is 
external to the camera. The camera includes a microphone to capture speech. The 

15 speech and digital image are communicated to the external processor as separate files. 
The processor converts the speech to text, as explained above, and combines the image 
and text into a single data file. This common file can be stored by the processor. This 
system can provide the benefits explained above without the physical limitations of a 
portable camera. As such, the system is better suited to a studio setting. Again, this 

20 process allows for the combination of text and image into a common file. 

Conclusion 

A digital camera has been described which includes a processor and a 
microphone. The camera captures audio speech and converts the speech to text. The 
25 text can be combined with a captured image to provide a composite file. The processor 
has been described as executing an instruction set which preforms the audio to text 
conversion. 

Although specific embodiments have been illustrated and described herein, it 
will be appreciated by those of ordinary skill in the art that any arrangement which is 
30 calculated to achieve the same purpose may be substituted for the specific embodiment 
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shown. This application is intended to cover any adaptations or variations of the present 
invention. Therefore, it is manifestly intended that this invention be limited only by the 
claims and the equivalents thereof. 
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What is claimed is: 

1 1. A camera comprising: 

2 a photo-sensitive array for capturing an image; 

3 a microphone; 

4 a memory; and 

5 a processor coupled to the photo-sensitive array, microphone and memory, the 

6 processor converts audio input provided by the microphone into text and stores the text 

7 in the memory. 

1 2. The camera of claim 1 wherein the photo-sensitive array comprises a charge 

2 coupled device, 

1 3. The camera of claim 1 wherein the processor stores the captured image as a 

2 digital data file in the memory. 

1 4, The camera of claim 1 wherein the processor stores the captured image and the 

2 text as a single digital data file in the memory. 

1 5. The camera of claim 1 further comprising an input control for activating the 

2 processor to capture audio input provided via the microphone. 

1 6. The camera of claim 5 wherein the input control is a switch. 

1 7. The camera of claim 5 wherein the input control is an audio conamand provided 

2 via the microphone to the processor. 
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18. A camera comprising: 

2 a photo-sensitive array for capturing an image; 

3 a microphone; 

4 a memory; and 

5 a processor coupled to the photo-sensitive array, microphone and memory, the 



6 processor converts captured audio input provided by the microphone into a digital text 

7 file and converts the captured image into a digital image file, and wherein the processor 

8 stores the digital image file and the digital text file as a single composite digital data file 

9 in the memory. 

1 9. The camera of claim 8 fixrther comprising an input control for activating the 

2 processor to capture audio input provided via the microphone, 

1 10. The camera of claim 9 wherein the input control is an audio command provided 

2 via the microphone to the processor. 

1 11. The camera of claim 8 wherein the processor fiirther stores the captured audio as 

2 a separate audio file in the memory. 

1 12. A method of operating a camera comprising: 

2 activating a shutter of the camera to capture a light image; 

3 converting the light image to digital image data; 

4 activating an audio input; 

5 capturing audio input; and 

6 converting the audio input into text data. 

1 13. The method of claim 12 further comprising storing the digital image data and the 

2 text data in a memory of the camera. 
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1 14. The method of claim 1 3 wherein the digital image data and the text data are 

2 stored in the memory of the camera as a composite digital file. 



1 15. A method of operating a camera comprising: 

2 activating a shutter of the camera to capture a light image using a photosensitive 

3 array; 

4 converting the light image to digital image data; 

5 activating an audio input; 

6 capturing audio input; 

7 converting the audio input into text data; and 

8 storing the text data and the digital image data as a single digital data file in a 



9 memory of the camera. 



1 16. A camera system comprising: 

2 camera having a photo-sensitive array for capturing an image, a microphone, and 

3 a memory; and 

4 an external processor coupled to the camera, the processor converts audio input 

5 provided by the camera into text and combines the text and the image provided by the 

6 camera into a common data file. 



1 17. The camera system of claim 16 further comprising an input control for activating 

2 the external processor to capture audio input provided via the microphone. 

1 18. The camera system of claim 17 wherein the input control is a switch. 

1 19. The camera system of claim 17 wherein the input control is an audio conmiand 

2 provided via the microphone to the processor. 
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1 20. A method of operating a camera system comprising: 

2 activating a shutter of the camera to capture a Ught image using a photosensitive 

3 array; 

4 converting the Hght image to digital image data; 

5 activating an audio input; 

6 capturing audio input; 

7 converting the audio input into text data using an external processor; and 

8 storing the text data and the digital image data as a single digital data file. 
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Abstract of the Disclosure 

A digital camera includes a processor and a microphone. The camera captures 
audio speech and converts the speech to text. The text can be combined with a captured 
image to provide a composite file. The processor executes an instruction set which 
preforms the audio to text conversion. Text conversion is activated using either a user 
activated input control, or through voice commands. 
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Attorney Docket No 884.1 66US1 

SCHWEGMAN, LUNDBERG, WOESSNER & KLUTH, P.A. 

United States Patent Application 

COMBINED DECLARATION AND POWER OF ATTORNEY 

As a below named inventor I hereby declare that: my residence, post office address and citizenship are as 
stated below next to my name; that 

I verily believe I am the original, first and sole inventor of the subject matter which is claimed and for which a 
patent is sought on the invention entitled: SPEECH TO TEXT CAPTTONTNG FOR DTGTTAL CAMERA 

The specification of which is attached hereto. 

I hereby state that I have reviewed and understand the contents of the above-identified specification, including 
the claims, as amended by any amendment referred to above. 

I acknowledge the duty to disclose information which is material to the patentability of this application in 
accordance with 37 C.F.R. § 1 .56 (attached hereto). I also acknowledge my duty to disclose all information known to 
be material to patentability which became available between a filing date of a prior application and the national or 
PCflntemational filing date in the event this is a Continuation-In-Part apphcation in accordance with 37 C.F.R. 
§!.#)• 

I hereby claim foreign priority benefits under 35 U.S.C. §1 19(a)-(d) or 365(b) of any foreign apphcation(s) for 
pateM or inventor's certificate, or 365(a) of any PCI intemational application which designated at least one country 
oth^ithan the United States of America, listed below and have also identified below any foreign apphcation for 
pateB or inventor's certificate having a filing date before that of the apphcation on the basis of which priority is 
claimed: 

No mch claim for priority is being made at this time. 

C| I hereby claim the benefit under 35 U.S.C. § 1 19(e) of any United States provisional application(s) hsted 
belog^: 

No such claim for priority is being made at this time. 

I hereby claim the benefit under 35 U.S.C, § 120 or 365(c) of any United States and PCT intemational 
application(s) hsted below and, insofar as the subject matter of each of the claims of this apphcation is not disclosed 
in the prior United States or PCT intemational apphcation in the manner provided by the first paragraph of 35 U.S.C. 
§ 1 12, 1 acknowledge the duty to disclose material information as defined in 37 C.F.R. § 1.56(a) which became 
available between the filing date of the prior application and the national or PCT intemational filing date of this 
apphcation: 
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§1.56 Duty to disclose information material to patentability. 

(a) A patent by its very nature is affected with a public interest. The public interest is best served, and the most effective patent 
examination occurs when, at the time an application is being examined, the Office is aware of and evaluates the teachings of all information 
material to patentability. Each individual associated with the filing and prosecution of a patent application has a duty of candor and good 
faith in dealing with the Office, which includes a duty to disclose to the Office all information known to that individual to be material to 
patentability as defmed in this section. The duty to disclose information exists with respect to each pending claim until the claim is canceled 
or withdrawn from consideration, or the application becomes abandoned. Information material to the patentability of a claim that is canceled 
or withdrawn from consideration need not be submitted if the information is not material to the patentability of any claim remaining under 
consideration in the application. There is no duty to submit mformation which is not material to the patentability of any existing claim. The 
duty to disclose all information known to be material to patentability is deemed to be satisfied if all information known to be material to 
patentability of any claim issued in a patent was cited by the Office or submitted to the Office in the manner prescribed by §§ 1 .97(b)-(d) 
and 1.98. However, no patent will be granted on an apphcation in connection with which fraud on the Office was practiced or attempted or 
the duty of disclosure was violated through bad faith or intentional misconduct. The Office encourages apphcants to carefully examine: 

(1) prior art cited in search reports of a foreign patent office in a counterpart application, and 

(2) the closest information over which individuals associated with the filing or prosecution of a patent application believe any 
pending claim patentably defmes, to make sure that any material information contained therein is disclosed to the Office. 

(bj; Under this section, information is material to patentability when it is not cumulative to information akeady of record or being 
made„-_©f record in the application, and 

(1) It establishes, by itself or in combination with other information, a prima facie case of unpatentability of a claim; or 

^zt (2) It refutes, or is inconsistent with, a position the applicant takes in: 

" (i) Opposing an argument of unpatentability relied on by the Office, or 

;i| (ii) Asserting an argument of patentability. 

A piMa facie case of unpatentability is established when the information compels a conclusion that a claim is unpatentable under the 
preponderance of evidence, burden-of-proof standard, giving each term in the claim its broadest reasonable construction consistent with the 
specijication, and before any consideration is given to evidence which may be submitted in an attempt to establish a contrary conclusion of 
pateiifability. 

(c) Individuals associated with the filing or prosecution of a patent apphcation within the meaning of this section are: 

(1) Each inventor named in the application: 

(2) Each attorney or agent who prepares or prosecutes the application; and 

(3) Every other person who is substantively involved in the preparation or prosecution of the application and who is associated 
with the inventor, with the assignee or with anyone to whom there is an obligation to assign the application. 

(d) Individuals other than the attorney, agent or inventor may comply with this section by disclosing information to the attorney, 
agent, or inventor. 



